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ABSTRACT 


Structural  Health  Monitoring  (SHM)  sensor  systems  have  gained  much  interest  in  the 
aerospace  community  as  potentially  enabling  technologies  for  reducing  maintenance 
costs  while  maintaining  flight  safety  of  aging  structures.  Many  candidate  SHM 
technologies  are  essentially  damage  detection  systems,  i.e.  variations  of  well- 
established  NDI  methods  such  as  ultrasonics,  electro-magnetic  (eddy  current),  and 
acoustic  emission.  Still  others  employ  displacement  detection  such  as  strain-sensing 
and  dynamic  vibration  sensing.  Unfortunately,  the  path  for  transition  onto  United  States 
Air  Force  (USAF)  weapon  systems  is  not  well  defined.  Before  SHM  technologies  can 
be  qualified  for  fleet-wide  usage,  these  systems  must  demonstrate  their  capability  to 
meet  performance  requirements  in  terms  of  detection  capability,  reliability,  and  safety 
while  and  providing  an  economic  benefit.  Ultimately  the  qualification  program  must  be 
sufficiently  robust  to  support  an  airworthiness  certification  decision.  This  document 
provides  guidance  for  the  demonstration  and  qualification  of  permanently  mounted 
damage  detection  systems  intended  for  use  on  USAF  aircraft.  More  specifically,  this 
document  will  a)  focus  on  applications  where  damage  tolerance  principles  are  used  to 
manage  structural  integrity,  b)  propose  a  task-based  approach  for  system  qualification, 
and  c)  identify  existing  Air  Force  policies  that  define  the  path  for  modification 
management  and  obtaining  airworthiness  certification  approval. 
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1.0  INTRODUCTION 


The  United  States  Air  Force  (USAF)  is  facing  challenges  in  maintaining  an  increasingly 
aging  fleet.  Increasing  maintenance  and  inspection  requirements  result  in  extended 
maintenance  downtime  and  lower  mission  capable  rates.  To  address  these  issues,  the 
USAF  is  exploring  approaches  to  reduce  the  disassembly  and  downtime  required  to 
execute  invasive  structural  inspections  that  are  critical  to  maintaining  structural  integrity 
and  to  anticipate  maintenance  needs(1).  One  potential  solution  is  the  use  of  structural 
health  monitoring  (SHM)  to  provide  state  awareness  of  structural  health  and  usage. 
However,  before  these  technologies  can  be  embedded  in  aircraft,  their  performance, 
reliability,  impact  on  aircraft  safety,  and  economic  benefit  must  be  successfully 
demonstrated. 

This  paper  focuses  exclusively  on  defining  a  framework  for  qualifying  on-board  damage 
detection  systems,  from  this  point  forward  defined  as  Structural  Damage  Sensing  (SDS) 
systems.  SDS  systems  exist  as  pieces  of  the  overall  SHM  concept  (Figure  1 ) (1).  The 
entire  SHM  concept  is  not  limited  to  SDS  systems  but  is  more  broadly  defined  as  a 
system  utilizing  multiple  sensor  data  streams  such  as  loads  and  environment  (vibration, 
temperature,  corrosive  environments,  etc.)  in  conjunction  with  the  SDS  data  and  pre¬ 
existing  structural  models  to  predict  structural  health  through  a  processing  reasoner. 


Figure  2:  The  several  components  of  a  Structural  Health  Monitoring  concept 
including  the  Structural  Damage  Sensing  (SDS)  component. 


The  concept  of  structural  health  management  differs  from  structural  health  monitoring 
as  it  includes  data  collected  from  multiple  sources  including  findings  from  previous 
inspections,  manufacturing  &  repair  data,  other  engineering  data,  etc.  Structural  health 
management  includes  greater  prognostic  and  decision  making  capabilities  to  aid  in 
structural  life  prediction.  The  flowchart  in  Figure  2  illustrates  the  various  elements  that 
make  up  a  possible  version  of  an  overall  structural  health  management  model.  The 
SDS  piece  is  included  in  the  Sensors  box  within  the  Data  Collection  element. 

Therefore,  SDS  and  SHM  are  entirely  contained  within  and  exist  as  sub-functions  of  an 
overall  structural  health  management  concept. 
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Figure  2:  Elements  and  process  for  Structural  Health  Management.  The  SDS 
component  is  contained  within  the  Sensors  box  under  the  Data  Collection 
element.  Diagram  courtesy  of  The  Boeing  Corporation  and  AFRL/RBSI 

How  SDS  systems  are  qualified  and  implemented  within  this  structural  management 
methodology  greatly  depends  on  the  operational  principles  of  the  SDS  system.  Two 
basic  scenarios  are  envisioned: 

Category  I  Systems  -  Active  Systems:  Active  SDS  systems  use  on-board  continuous 
monitoring  where  the  inspection  result  is  monitored  and  interpreted  real-time  or  at 
relatively  short  intervals.  In  this  scenario  correlation  of  the  sensor  output  to  damage 
severity  is  required  to  establish  a  decision  threshold.  Moreover,  since  the  sensor  is 
constantly  monitored,  the  probability  of  a  miss  may  potentially  be  reduced  but  with  a 
corresponding  increase  in  false  positive  rates.  Of  course  this  advantage  is  only  realized 
if  the  system  is  installed  correctly  and  is  monitoring  the  region  where  a  flaw  occurs. 

High  false-positive  rates  could  negatively  impact  aircraft  availability  by  requiring  the 
aircraft  to  be  down  for  SDS  system  troubleshooting  and/or  for  “chasing”  imaginary 
damage.  On-board  installation  of  the  interrogating  instrumentation  as  well  as  an  on¬ 
board  power  is  assumed  to  be  required;  as  a  result,  flight-safety  certification  for  this 
category  becomes  significantly  more  complex. 

Category  II  Systems  -  Passive  Systems:  Passive  SDS  systems  use  a  “plug-and-play” 
approach  where  the  damage  sensor  interrogates  structural  only  at  a  prescribed  interval 
similar  to  the  tradition  NDI  methodologies.  Characterization  of  the  detection  capability  is 
critical  and  false  positive  rates  are  expected  be  lower  since  the  sensor  is  not  constantly 
monitored.  For  this  scenario,  it  is  assumed  that  onboard  installation  of  only  the  sensor 
and  cabling  is  required.  The  system  is  not  activated  in-flight  and  no  on-board  power  is 
required.  Because  of  the  limited  hardware  installation,  this  category  reduces  the 
complexity  for  airworthiness  certification. 


2 

Approved  for  public  release;  distribution  unlimited 


2.0  DEFINING  SDS  REQUIREMENTS 


Establishing  a  program  to  implement  SDS  systems  requires  a  clear  definition  of  the 
application.  There  are  several  considerations  that  must  be  addressed  when  formulating 
this  definition.  These  considerations  include,  but  are  not  limited  to,  structural 
configuration,  structural  variation,  usage  environment,  system  durability  requirements, 
configuration  management,  and  system  maintenance.  As  such,  a  Master  Requirements 
Document  (MRD)  must  be  generated  to  capture  the  application  specific  requirements 
and  drive  the  plan  formulation.  The  MRD  should  be  approved  by  the  weapon  system 
ASIP  Manager  and  Chief  Systems  Engineer  and  must  be  releasable  to  potential 
vendors  wishing  to  propose  an  SDS  system  or  technology. 

As  a  minimum  the  MRD  must  define  the  following  (7): 

1 .  Part  Geometry  -  Often  conveyed  through  production/engineering  drawings  and 
usually  developed  and  supplied  by  the  Original  Equipment  Manufacturer  (OEM). 
While  these  drawings  typically  define  specific  dimensional  information  regarding 
the  part  or  assembly,  they  sometimes  lack  information  necessary  to  fully  define 
the  local  structural  interfaces,  geometric  interference,  manufacturing  variability 
and  access.  It  is  often  necessary  to  “lay-hands”  on  the  structural  details  to  guide 
SDS  system  development  decisions. 

2.  Part  Material  -  The  material  description  must  include,  in  the  case  of  metallic 
structure,  the  alloy  type  and  heat  treatment  or  temper  condition,  and  may  require 
a  description  of  any  surface  treatments  including  coatings  or  plating  and 
thicknesses.  In  addition,  material  details  may  be  required  for  other  structure 
located  adjacent  to  the  region  of  interest,  and,  if  fasteners  are  located  within  the 
region,  the  fastener  type  and  material  composition  must  also  be  provided. 

3.  Flaw  Location  and  Orientation  -  A  clear  definition  of  both  the  expected  flaw 
location  and  orientation  is  required.  This  information  may  be  available  in  the 
form  of  damage  tolerance  analysis,  and  fatigue  test  results  (subcomponent, 
component,  or  full  scale).  The  reliability  of  this  information  is  greatly  dependent 
on  the  fidelity  of  the  test  program  and  analysis  and  will  often  evolve  as  the 
aircraft  system  matures.  These  details  will  focus  the  development  and  testing  of 
SDS  systems,  providing  the  optimum  opportunity  to  maximize  the  detection 
capability. 

4.  Effectivitv/Confiquration  Changes  -  A  list  of  affected  aircraft  or  systems  by  tail  or 
serial  number  is  also  required.  This  information  should  include  a  description  of 
any  deviations  or  configuration  changes  in  component  design,  including 
variances  in  any  of  the  items  described  above.  Such  an  accounting  provides  a 
level  of  assurance  that  all  affected  systems  are  inspected  and  that  SDS 
processes  are  appropriately  adjusted  to  compensate  for  known  variances. 

5.  Assembly  -  In  many  cases,  the  structural  assembly  is  also  conveyed  through 
production/engineering  drawings.  This  description  must  define  the  geometry  and 
composition  of  mating  components  and  how  these  mating  components  are  joined 
to  the  component  under  interrogation.  This  assembly  defines  the  boundary 
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conditions  under  which  the  SDS  system  must  reliably  function.  The  assembly 
configuration  can  affect  the  sensor  design  and  placement.  Adjacent  components 
or  local  part  geometry  could  result  in  a  confounding  SDS  response  that  could  be 
misinterpreted  as  a  defect  or  even  mask  a  defect. 

6.  Structural  Variability  -  Potential  structural  variability  that  could  affect  the  reliability 
or  repeatability  of  an  SDS  system  should  be  defined.  Sources  of  variability 
include  but  are  not  limited  to  variations  in  structural  faying  surface  interfaces, 
coating  systems,  or  part  configuration  often  due  in  large  part  to  production 
changes,  repairs  or  deterioration  of  materials  over  time  that  are  not  otherwise 
captured  by  the  aircraft  configuration  control  process.  Records  of  these 
deviations  may  include  any  documented  nonconformance  as  accepted  by 
Materials  Review  Boards.  In  addition,  standard  repair  processes  that  have  been 
applied  to  fielded  components  that  will  be  encountered  during  inspection  should 
also  be  documented.  The  development  of  an  SDS  demonstration  program 
should  account  for  structural  variability  within  the  test  matrix. 

7.  Access  for  Installation/Stay-Out  Zones  -  Points  of  access  for  installation  and 
repair  of  the  SDS  system  must  be  identified.  This  description  should  include:  a) 
panels  or  doors  that  can  be  removed  which  may  provide  access  and  facilitate 
system  installation,  b)  description  of  local  structure  or  subsystems  that  may 
hinder  access  (particularly  on  low  observable  platforms),  c)  areas  that  can  be 
used  for  cable  routing  or  other  system  subcomponents,  d)  aircraft  systems  that 
may  be  affected  by  SDS  hardware,  e)  regions  that  cannot  be  used  to  mount  SDS 
system  subcomponents  (stay-out  zones). 

8.  Capability  ( aNm )  -It  is  good  practice  to  provide  both  a  goal  and  threshold  aWD/ 
value.  Historically  the  value  of  aNDi  has  been  established  as  the  a90/95  value 
ideally  determined  statistically  using  the  empirical  methods  defined  in  MIL-HDBK- 
1823A  (5).  The  a90/9 5  is  an  estimate  of  the  crack  size  that  will  be  detected  90%  of 
the  time  with  a  statistical  confidence  of  95%.  The  goal  value  is  the  detection 
capability  that  may  be  very  challenging  to  meet  but  would  result  in  inspection 
intervals  that  provide  an  economic  or  maintenance  benefit  to  the  program.  The 
threshold  value  is  the  detection  capability  that  cannot  be  exceeded  and  if 
exceeded  would  pose  an  unacceptable  economic  or  availability  burden  to  the 
program.  For  example,  the  inability  to  achieve  the  threshold  size  would  require 
aircraft  groundings  or  drive  structural  replacement  or  modification  actions.  The 
goal  and  threshold  values  should  be  used  to  develop  the  SDS  demonstration 
experiment.  In  addition,  these  values  should  be  used  to  develop  SDS 
interrogation  intervals.  Detection  capability  estimates  for  standard  inspection 
processes  are  summarized  in  Structures  Bulletin  EN  SB-008-012,  Nondestructive 
Inspection  Capability  Guidelines  for  United  States  Air  Force  Aircraft  Structures. 

9.  False  Positive  Rates  -  False  positives  (also  known  as  false  alarms)  can  present 
a  significant  economic  and  availability  burden  if  not  appropriately  controlled  as 
they  can  drive  costly  and  intrusive  structural  disassembly.  Therefore  the 
maximum  rate  of  false  positives  must  also  be  defined.  It  is  recommended  that 
false  positive  rate  should  not  exceed  1x1  O'4  per  inspection  cycle  for  SDS 
systems. 
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10.  Durability:  System  durability  requirements,  in  terms  of  minimum  mean-time- 
between  failure  rates,  should  be  defined.  Failure  rates  must  be  sufficiently  low  to 
support  the  maintenance  concept  and  provide  long  term  monitoring  without  the 
need  for  invasive  maintenance  or  repair  of  the  monitoring  system.  Durability  will 
be  discussed  in  more  detail  later  in  this  paper. 

1 1 .  Usage  Environment:  The  usage  environment  includes  but  is  not  limited  to 
temperature  profiles,  humidity,  fuel,  hydraulic  fluid  or  chemical  exposure,  strain 
and  vibration.  A  definition  of  this  environment  will  drive  the  design  of 
environmental  and  durability  testing  and  the  qualification/airworthiness 
requirements. 

12.  Other  Requirements:  The  MRD  should  clearly  define  other  aircraft  specific 
requirements.  These  may  include  maximum  system  weight  and  size,  power 
requirements,  etc.  Development  of  the  MRD  should  be  closely  coordinated  with 
the  appropriate  system  engineering  authority  and  safety. 

Vendors  submitting  proposals  must  define  the  candidate  technology’s  maturity  level  (i.e. 
technology  readiness  level)  and  ability  to  meet  each  of  the  detailed  requirements 
defined  by  the  MRD.  These  proposals  should  be  accompanied  by  technical  data  (e.g. 
test  results,  drawings,  documentation  of  previous  applications)  to  support  the 
performance  claims.  Sufficient  evidence  must  be  provided  to  the  Program  Manager  that 
a  proposed  technology  is  applicable  and  is  sufficiently  mature  to  justify  a  qualification 
test  program. 
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3.0  IMPLEMENTING  SDS  FOR  DAMAGE  TOLERANCE  APPLICATIONS 


Implementation  of  SDS  systems  into  USAF  aircraft  requires  an  understanding  of 
processes  used  by  the  USAF  to  manage  structural  integrity  over  the  aircraft’s  lifetime. 
The  Aircraft  Structural  Integrity  Program  (ASIP),  required  by  MIL-STD-1530C  (2) 
employs  damage  tolerance  (DT)  as  the  methodology  for  managing  the  safety,  repair, 
modification  and  inspection  of  safety-of-flight  aircraft  structures.  Damage  tolerance 
(DT)  is  defined  as  the  ability  of  a  structure  to  retain  its  required  strength  for  a  period  of 
unrepaired  usage  after  the  structure  has  sustained  specific  levels  of  fatigue  cracking, 
corrosion  or  discrete  damage.  In  simple  terms,  maintenance  and  inspection  actions  are 
scheduled  such  that  the  damage  is  detected  and  repaired  before  it  reaches  a  size  that 
presents  an  unacceptable  risk  to  structural  safety  or  that  cannot  be  economically 
repaired. 

When  DT  is  implemented  for  slow  crack  growth  metallic  structures,  initial  inspections 
are  typically  based  on  “rogue”  initial  flaw  size  assumptions  and  slow  crack  growth. 

When  inspections  are  prescribed,  it  has  been  USAF  policy  to  require  the  initial 
inspection  to  occur  at  a  point  in  time  (T i )  which  corresponds  to  half  the  crack  growth  life 
associated  with  growing  the  crack  from  its  initial  rogue  flaw  size  (a0)  to  the  critical  crack 
size,  (acr)  (Figure  3)(3)(4l  The  critical  flaw  size  is  the  size  at  which  immediate  and 
catastrophic  failure  of  the  structure  can  occur  if  the  structure  experiences  limit  load. 


Figure  3.  Slow  crack  growth  curve  from  initial  rogue  flaw  (a0)  at  time  zero  to 
critical  crack  size  ( acr )  at  the  time  for  catastrophic  failure  (7».  Per  USAF  ASIP 
policy,  the  initial  inspection  is  required  at  a  time  ( T1 )  occurring  at  half  the  crack 
growth  life.  The  capability  of  an  inspection  system  may  be  defined  by  the  length 
of  crack  that  such  a  system  might  miss  (aNDi) 
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Recurring  inspection  intervals  are  established  at  half  the  time  required  for  a  crack  to 
grow  from  the  largest  crack  length  that  may  be  assumed  to  be  missed  (i.e.,  the  NDI 
capability)  during  an  inspection,  aNDi,  to  the  critical  size  acr  (Figure  4).  If  no  crack  is 
detected  during  an  inspection,  the  crack  size  is  analytically  “reset”  to  aNDi{\-e-,  the 
structure  is  assumed  to  have  a  crack  of  length  aND/ present)  and  the  next  inspection 
interval  is  calculated  as  before.  Recall  that  awo/ signifies  an  estimate  of  the  detection 
capability  an  inspection  system  typically  established  at  the  a90/95 crack  size.  Again,  the 
390/95  is  a  statistical  estimate  of  the  crack  size  that  will  be  detected  90%  of  the  time  with 
a  statistical  confidence  of  95%. 

The  ability  to  define  the  damage  detection  capability  of  potential  SDS  solutions  is  critical 
for  successful  implementation.  Therefore  it  is  critical  that  SDS  systems  demonstrate  the 
capability  to  reliably  detect  damage  such  that  the  probability-of-miss  (POM)  is  very  low 
(i.e.  the  probability  that  a  missed  crack  of  significant  size,  acr-miss  will  not  grow  to  failure 
before  the  next  inspection  interval).  Estimating  detection  capability  is  not  a  trivial  task 
and  requires  considerable  expertise  in  statistics  and  design  of  experiments  to 
successfully  accomplish.  The  statistical  methods  defined  in  MIL-HDBK-1823A  (5)  should 
always  be  used. 


Figure  4.  The  slow  crack  growth  curve  from  a  post-inspection  based  ’’reset” 
crack  size  (aNDi)  growing  from  time  (Ti)  to  the  critical  crack  size  (acr)  at  time  (T3) 
defines  the  time  for  the  next  inspection  (at  time  T2)  Also  shown  in  the  figure  is 
the  critical-miss  crack  size  (acr-miss)  associated  with  a  crack  that  will  grow  to 
failure  before  the  following  inspection  at  time  T3  if  missed  during  the  next 

inspection  at  time  T2. 

The  DT  methodology,  first  established  in  the  1970s(3),  has  evolved  over  the  years  but 
remains  focused  on  establishing  structural  configurations  and  stress  allowables  to 
control  and  mitigate  the  potential  for  structural  failures,  throughout  the  design  lifetime, 
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as  a  result  of  crack-like  damage.  It  is  important  to  note  that  USAF  ASIP  policy  strives  to 
address  manufacturing  and  design  issues  early  in  a  program’s  life-cycle  to  preclude 
costly  structural  retrofits  or  recurring  inspections  required  to  maintain  structural  safety. 
The  current  version  of  the  Department  of  Defense  Joint  Service  Guide  Specification, 
JSSG  2006  (6)  clearly  states  in  §3.2.14.5  “By  design,  the  airframe  structure  shall  not 
require  inspection  during  the  service  life  specified  in  §3.2. 14".  However,  if  it  is 
determined  that  cracks  will  likely  occur  in  service  and  may  propagate  to  failure  before 
two  design  lifetimes,  then  the  program  must  consider  the  business  case  justification  for 
one  or  more  of  the  following  solution  options: 

Option  1 :  Redesign  the  part  and  change  the  configuration  during  production 
Option  2:  Redesign  the  part  and  retrofit  during  sustainment 
Option  3:  Develop  an  inspection  and  repair  program  to  ensure  that  cracks  do  not 
grow  to  failure  or  beyond  an  economically  repairable  size  during  the  expected 
service  life  of  the  airframe. 

Historically,  design  shortfalls  are  often  discovered  during  full-scale  fatigue  testing  very 
late  in  the  design  and  development  phase.  This  challenge  is  often  compounded  by 
some  level  of  concurrency  between  production  and  the  test  program.  As  a  result,  the 
ability  to  implement  an  Option  1  solution  for  early  production  airframes  is  extremely 
limited  and  Option  2  and  3  solutions  are  often  employed.  Consequently,  the  ability  to 
embed  an  SDS  solution  early  in  production  is  severely  limited  or  is  not  borne  out  by  the 
typical  business  case  which  often  leads  to  structural  retrofit.  SDS  systems  may, 
however,  provide  a  technically  viable  and  cost  effective  alternative  as  an  Option  3 
solution. 

Structural  inspection  requirements  are  defined  through  a  combination  of  analysis,  full- 
scale  fatigue  testing,  and  field  findings  and  are  often  refined  and  focused  to  specific 
structural  locations  and  details.  It  is  rare  for  individual  inspection  requirements  to 
encompass  large  areas.  These  individual  “control  points”  are  typically  defined  by  an 
airframe’s  Force  Structural  Maintenance  Plan  (FSMP)  that  clearly  describes  each 
point’s  structural  location,  required  inspection  capability  and  required  inspection  interval. 
These  requirements  are,  for  the  most  part,  well  defined  and  provide  a  foundation  for 
SDS  system  performance  testing. 
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4.0  SDS  TRANSITION  PATH 


A  successful  SDS  solution  must  provide  a  robust  and  reliable  capability  to  protect 
structural  safety  while  providing  cost  savings  over  traditional  inspection  processes.  The 
decision  to  implement  must  be  supported  by  a  rigorous  transition  program  that  includes 
the  following  tasks: 

Task  1:  Demonstration 
Task  2:  Structural  Risk  Assessment 
Task  3:  Cost  Benefit  Analysis 
Task  4:  Airworthiness  Certification 
Task  5:  Implementation  and  Tracking 

Taskl:  Demonstration 

The  basis  of  a  demonstration  program  is  the  development  of  an  SDS  demonstration  test 
plan  that  addresses  the  following  four  critical  factors: 

-  Capability 

-  Durability 

-  Installation/Supportability 

-  Safety 

A  robust  demonstration  program  should  use  a  multi-phased,  building-block  approach 
with  clearly  defined  exit  criteria  for  each  phase  documented  in  a  SDS  Demonstration 
Test  Plan.  Typically  such  a  demonstration  program  will  begin  with  basic  laboratory 
testing  that  will  eventually  lead  to  on-aircraft  flight  test  (Figure  5).  Each  phase  must 
address  various  aspects  of  the  four  critical  factors  (detection  capability,  durability, 
installation/supportability,  safety)  with  a  successful  outcome  supporting  a  decision 
milestone  to  move  to  the  next  phase.  It  is  intended  that  this  approach  should  be  tailored 
based  on  the  criticality  of  the  application  and  the  maturity  of  the  proposed  SDS  system. 
Both  the  increase  in  the  criticality  of  the  intended  application  and  the  lack  of 
demonstrated  maturity  of  a  proposed  SDS  system  will  drive  more  extensive 
development  and  testing. 

To  address  this  scale-up  approach  a  demonstration  program  should  include  the 
following  phases: 

Phase  I:  Laboratory  Testing 

Phase  II:  Simulated  Relevant  Environment 

Phase  III:  Systems  Level  Test 

Phase  IV:  Installation  Validation 

Phase  V:  Flight  Test 
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Figure  5  illustrates  this  building-block  approach  with  decision  gates  at  the  conclusion  of 
each  phase.  The  primary  objective  of  the  demonstration  test  program  is  to  generate 
sufficient  evidence  that  the  proposed  SDS  system  meets  the  MRD  requirements  prior  to 
transitioning  to  Task  2.  Each  demonstration  phase  must  be  structured  to  address 
specific  objectives  described  as  follows: 


Figure  5:  Task  1  -  Notional  Demonstration  Test  Program  for  SDS  Systems 


Demonstration:  Phase  I  -  Initial  Laboratory  Testing: 

Initial  laboratory  testing  provides  the  initial  proof-of-concept  or  feasibility  with  a  primary 
focus  on  demonstrating  compatibility  to  the  intended  application  and  capability  of  the 
system  to  detect  type  and  size  of  damage  within  representative  test  samples.  For  an 
SDS  system  to  successfully  meet  the  requirements  for  Phase  I  testing,  the  following 
objectives  must  be  met: 

•  Demonstration  of  compatibility  with  the  intended  application.  This  includes,  but  is 
not  limited  to,  compatibility  with  the  simulated  structural  configuration,  geometry  and 
materials.  The  simulated  structure  should  be  manufactured  using  the  same 
processes  defined  by  the  design. 
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•  Demonstration  of  capability  to  detect  damage  as  required  by  the  MRD  within  the 
defined  environment  on  specimens  representing  the  material  and  configuration  of 
the  component  of  interest. 

•  Demonstration  of  the  SDS  system  calibration  approach  and  procedures. 

•  The  hardware  and  interface  used  for  monitoring  the  SDS  system  response  must 
demonstrate  the  ability  to  provide  a  clear  and  unambiguous  indication  of  damage. 
The  damage  response  should  continuously  increase  with  increasing  levels  of 
damage  until  a  saturated  response  is  encountered.  The  signal  response  should  be 
directly  correlated  to  damage  severity. 

•  The  ability  to  threshold  the  SDS  signal  response  to  indicate  a  discrete  level  of 
damage  must  also  be  demonstrated.  The  threshold  level  must  be  selected  to  detect 
the  type  and  size  of  interest  as  defined  by  the  MRD  without  false-positives. 

Basic  feasibility  testing  can  consist  of  mounting  SDS  sensors  to  representative 
specimens  and  cyclically  loading  the  specimens  to  generate  and  grow  fatigue  damage. 
Preliminary  testing  may  involve  the  use  of  simulated  defects  (e.g.  electro-discharge 
machined  (EDM)  notches,  simulated  disbonds/delaminations)  to  represent  damage  but 
should  progress  to  use  of  cyclically  loaded  fatigue  damaged  specimens.  The  loading 
spectrum  used  for  fatigue  propagation  should  be  based  on  the  anticipated  on-aircraft 
load  environment;  however,  higher  load  rates  may  be  required  for  economy.  Test 
specimens  must  be  manufactured  from  the  same  material,  alloy,  heat  treat  and  possess 
a  similar  microstructure  as  the  intended  application.  The  sample  design  sufficiently 
complex  (contain  stiffeners,  fastener  holes,  tapers,  curves,  etc.  as  appropriate)  to 
represent  the  intended  application  but  may  not  require  the  detailed  replication  of  aircraft 
structure  geometry  or  assembly. 

Demonstration:  Phase  II  -  Simulated  Relevant  Environment: 

Building  upon  the  Phase  I  milestones,  the  goal  for  this  phase  is  to  demonstrate  the 
system  detection  capability  to  sense  and  reliably  identify  relevant  damage  on  structures 
in  a  relevant  environment.  A  relevant  environment  is  defined  as  test  conditions  that 
closely  simulate  the  load  spectrum  when  the  test  coupons  are  exposed  to  an 
environment  similar  to  the  intended  application.  Conditions  that  may  have  to  be 
simulated  include  vibration,  temperature,  pressure,  and  exposure  to  moisture  or  aircraft 
fluids  (hydraulic  fluids,  fuel,  greases).  The  test  samples  should  represent  the  intended 
application  in  terms  of  geometry,  material,  and  assembly,  including  boundary 
conditions.  In  addition,  the  simulated  environmental  testing  should  mirror  actual  in- 
service  conditions  to  the  greatest  extent  possible.  The  test  duration  should  be  at  least 
twice  the  intended  monitoring  period  of  the  SDS  system.  For  example,  if  the  structure 
will  be  monitored  by  the  SDS  system  between  depot  maintenance  cycles  and 
conventional  NDI  will  take  place  during  the  depot  maintenance  then  the  test  duration 
should  replicate  at  least  two  depot  cycles. 
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The  objectives  of  this  phase  include: 

•  Demonstration  of  the  capability  of  the  SDS  system  to  provide  accurate 
detection  information  without  performance  drift  in  a  simulated  relevant 
environment.  An  initial  estimate  of  system  durability  in  terms  of  equivalent 
flight  hours  should  result. 

•  Demonstration  of  the  long-term  integrity  of  sensor,  wiring  and  hardware 
mounting  and/or  bonding  to  the  structure. 

•  Demonstration  of  the  sensing  modality  and  full  characterization  of  sensor  response 
to  damage  severity  within  the  relevant  environment. 

•  Demonstration  of  the  capability  to  reject  relevant  damage  well  above  the  system  or 
environment  induced  noise.  Any  anomalies  in  the  results  must  be  explained  with 
physical  concepts. 

•  Validation  of  the  capability  to  reliably  detect  damage  in  terms  of  probability  of 
detection  as  defined  by  the  MRD  in  a  relevant  environment. 

The  SDS  sensors  and  systems  must  survive  in  a  relevant  environment  for  at  least  twice 
the  maximum  intended  monitoring  period  of  the  SDS  system  appropriate  for  the 
particular  application(s).  An  initial  assessment  of  the  possible  sensor  failure  modes, 
mean-time-between  failure  (MTBF)  rates  and  incorporation  of  design  improvements  or 
mitigation  strategies  should  result. 

Table  1:  Conditions  that  should  be  considered  during  simulated  relevant 

environmental  testing  (8). 


Environmental  Conditions 

High/Low  Temperature  (warm  climate,  altitude  temperature  ) 

Thermal  Shock  (engine  exhaust  blast) 

Humidity 

Fluid  Susceptibility  (jet  fuel,  hydraulic  fluid) 

Altitude/Low  Pressure 

Static  loading  and  strain  (fuel/weapons/cargo  weight) 

Fatigue  cycle  loading  (compressive  and  tensile) 

High  and  low  velocity  impact  loading/unloading  (bird  strikes,  fuel,  missile  firings) 

Vibration  (engine,  aero  or  acoustic  vibration) 

Exposure  to  slipstream  (for  sensors  placed  on  outer  mold  line) 

Electromagnetic  interference  (communications,  avionics,  flight  systems) 

Demonstration:  Phase  III  -  System  Level  Test:  Once  the  demonstration  test  program 
progresses  to  Phase  III,  testing  shifts  to  evaluations  on  complex  structure  that  mirrors 
the  structural  geometry,  materials,  scale  and  boundary  conditions  of  the  target  structure. 
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The  loads  and  environment  must  replicate  the  target  mission  profiles.  Demonstration 
on  full  scale  fatigue  test  articles  is  ideal.  This  may  also  include  a  subset  of  the  simulated 
environments  listed  in  Phase  II.  As  with  Phase  II,  the  test  duration  should  be  at  least 
twice  the  maximum  intended  monitoring  period  of  the  SDS  system. 

The  objectives  of  this  phase  include: 

•  Demonstration  of  the  capability  to  reliably  detect  damage  on  structure  representing 
the  target  structure  including  the  geometry,  scale,  materials  and  assembly 
interfaces. 

•  Demonstration  of  the  SDS  system  monitoring  scheme. 

•  Demonstration  and  verification  of  the  in-situ  calibration  procedures. 

•  Demonstration  and  verification  of  SDS  system  fault  sensing  capability. 

•  Identify  any  failure  modes  and  mitigation  strategies  not  previously  encountered  in 
Phase  II  and  establish  statistical  basis  for  mean-time-between-failure  (MTBF) 
estimates 

In  the  previous  testing  phases,  the  feasibility  of  the  SDS  installation,  capability  and 
durability  was  explored.  Phase  III  builds  on  previous  testing  by  incorporating  the 
lessons  learned  from  previous  phases  through  improvements  in  SDS  hardware, 
monitoring  schemes,  data  collection,  calibration  and  installation  processes.  Preparation 
for  Phase  III  requires  careful  engineering  to  ensure  that  all  potential  failure  modes  are 
considered  and  addressed. 

The  experimental  design  for  Phase  III  must  ensure  that  the  number  of  sensors,  control 
points  and  test  duration  are  sufficient  to  provide  a  statistically  relevant  estimate  of  SDS 
system  MTBF.  The  extended  test  duration  also  provides  an  opportunity  to  address  SDS 
system  variability  and  calibration  overtime. 

At  the  conclusion  of  this  testing  phase,  the  SDS  system  calibration  and  monitoring 
procedures  should  be  sufficiently  mature,  potential  SDS  system  failure  modes  should  be 
well  understood  and  addressed,  mounting/bonding  processes(9)  should  be  established 
and  all  potential  impacts  of  installation  to  aircraft  system  safety  addressed.  At  this  stage 
the  SDS  must  also  have  demonstrated  validated  detection  capability  as  required  by  the 
MRD  with  the  sources  of  potential  of  false-positives  understood  and  controlled. 

Demonstration:  Phase  IV  -  System  Installation  and  Validation: 

The  purpose  of  the  System  Installation  and  Validation  phase  is  to  exercise  the 
documented  installation  processes  and  identify  barriers  that  may  prevent  the  successful 
installation  of  an  SDS  system. 
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The  objectives  of  this  phase  include: 

•  Demonstration  and  validation  of  the  installation  procedures,  mounting  and/or 
bonding  processes,  wire  and  cable  runs  and  monitoring/access  points. 

•  Develop  and  validation  of  initial  installation,  monitoring  and  maintenance 
procedures  to  be  used  by  maintainers. 

•  Top-to-bottom  evaluation  of  the  SDS  system  safety  for  potential  hazards  (e.g.  EMI 
interference,  foreign  object  debris  potential,  intrinsic  safety  in  fuel  vapor  environment, 
electrostatic  discharge,  etc.) 

This  phase  differs  from  the  previous  phases  in  that  it  involves  the  installation  of  the  final 
SDS  system  configuration  onto  a  representative  airframe.  The  preference  is  to 
demonstrate  the  installation  on  a  non-flight  worthy  airframe,  such  as  a  maintenance 
trainer,  to  ensure  maximum  flexibility  in  resolving  installation  challenges. 

Potential  impacts  to  aircraft  safety  and  potential  system  interactions  must  be  evaluated 
using  the  guidance  of  DoD  MIL-HDBK-516B  Airworthiness  Certification  Criteria ”(10). 
Effects  on  the  basic  aircraft  functions  and  potential  interaction  with  subsystems  must  be 
addressed.  Potential  interactions  include  electromagnetic  interference  (EMI)(11),  effects 
on  avionics  and  communications,  power-supply  distribution  and  interactions.  To  that 
end,  ground  based  testing  of  the  SDS  system  prior  to  installation  on  an  airworthy  aircraft 
is  desirable. 

At  the  end  of  this  stage  all  installation  requirements  and  processes  should  be 
demonstrated  and  validated,  and  all  potential  system  interactions  and  safety 
considerations  must  be  evaluated,  addressed  and  verified. 

Demonstration:  Phase  V  -  Flight  Test:  The  flight  test  phase  marks  the  installation  and 
demonstration  of  the  mature  SDS  system  on  a  flight  worthy  aircraft.  Approval  for  flight 
test  is  gained  using  the  temporary  modification  process  (type  T2  modification)  defined 
by  AFI  63-131 ,  Modification  Program  Management (12),  with  the  technical  justification  for 
this  modification  provided  by  the  results  of  the  previous  demonstration  phases.  This 
phase  will  typically  involve  a  limited  number  of  aircraft;  therefore,  in  order  to  gain 
maximum  information  about  system  reliability,  mounting  and  monitoring  it  is 
recommended  that  multiple  sensors  and  sensor  systems  be  used.  To  maintain  flight 
safety  during  the  demonstration  program  the  current  inspection  requirements 
must  be  performed  in  parallel  to  SDS  monitoring. 

The  objectives  of  the  flight  test  demonstration  phase  include: 

•  Validation  of  field  or  depot  installation  processes  and  procedures 

•  Validation  of  the  data  capture,  monitoring  and  analysis  processes 
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•  Validation  of  SDS  system  reliability  in  intended  environment. 

•  Verification  of  the  reliability  of  the  SDS  system  in  terms  of  failure  rates  and  false¬ 
positive  rates. 

•  Identification  of  training  requirements  for  SDS  system  installation,  maintenance 
and  monitoring. 

•  Assessment  of  any  unforeseen  system  interactions. 


Task  2  -  Risk  Assessment 

A  successful  Task  1  demonstration  program  should  have  addressed  the  following: 

-  What  damage  type,  size  and  location  will  be  reliably  detected? 

-  What  is  the  system  capability  in  terms  of  probability  of  detection  and  probability 
of  miss? 

-  What  is  the  expected  rate  of  false  positive  indications? 

-  What  mean-time-between  failure  rate?  (i.e.  What  is  the  expected  service  life  of 
the  sensor  System?) 

The  first  two  questions  addresses  the  risk  to  structural  safety  while  the  third  and  fourth 
questions  drive  economic  risk  in  terms  of  unanticipated  maintenance  costs  and  impacts 
to  aircraft  availability.  If  these  questions  have  been  satisfactorily  answered,  then  a  risk- 
based  decision  must  be  made. 

An  SDS  system  that  is  used  to  monitor  fatigue  crack  growth  on  fracture-critical 
components  must  demonstrate  the  capability  to  reduce  risk  of  catastrophic  structural 
failure  while  providing  an  economic  benefit  over  convention  NDI  methodologies.  For 
safety  of  flight  (SoF)  structural  locations  a  risk  assessment  must  be  performed  to 
establish  that  the  candidate  SDS  system  will  maintain  a  probability  of  loss  of  an  aircraft 
(PLOA)  below  1x1  O'7  per  flight.  If  the  SDS  system  cannot  maintain  a  PLOA  below 
this  level,  then  the  system  cannot  replace  the  current  inspection  program. 

This  risk  assessment  must  be  based  on  a  credible  demonstration  of  the  SDS  system’s 
detection  capability  and  the  systems  reliability  in  addition  to  the  information  obtained  in 
Task  1 .  The  responsibility  for  conducting  this  risk  assessment  falls  to  the  responsible 
engineering  authority.  In  addition  to  a  risk  assessment,  the  risk  must  be  accepted 
by  the  appropriate  authority  as  called  out  in  Airworthiness  Bulletin  AWB-013A. 

If  the  risk  is  unacceptably  high,  and  not  accepted  at  the  appropriate  level,  then  various 
risk  mitigation  options  are  available  such  as  use  of  the  SDS  system  in  combination  with 
limited  NDI  (i.e.  -  every  other  inspection  interval)  or  use  of  an  SDS  system  to  provide 
expanded  coverage  at  specified  control  points  in  addition  to  the  focused  NDI  program  or 
to  monitor  fail-safe  structures  for  impending  failure. 
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Task  3  -  Cost  Benefit  Analysis 

The  decision  to  implement  an  SDS  system  will  ultimately  hinge  on  the  economic 
benefits  the  solution  provides  both  in  terms  of  cost  avoidance  and  system  availability. 
The  Program  Manager  must  develop  formal  cost  estimates  to  implement  the  proposed 
modification.  Approaches  for  developing  these  cost  estimates  are  defined  in  AFPD  65- 
5,  Cost  and  Economics^3}  AFPD  65-5  requires  estimate  include  all  life  cycle  costs 
including  recurring  cost  (e.g.  operation,  maintenance  and  monitoring)  and  non-recurring 
costs.  Non-recurring  costs  include  development  (laboratory  and  flight  test),  production, 
and  engineering  efforts  to  establish  airworthiness  certification  in  accordance  with  Air 
Force  Policy  Directive  (AFPD)  62-6,  USAF  Airworthiness}^  and  the  identification  and 
assessment  of  all  applicable  criteria  in  accordance  with  MIL-HDBK-516B.  Cost 
estimates  for  temporary  modifications  to  support  activities  such  as  flight  test  must  also 
include  host  system  de-modification.  If  the  program  is  determined  to  be  a  low  cost 
modification  (i.e.  less  than  $2  million)  then  preparation  of  a  formal,  detailed  cost 
estimates  may  not  be  required.  Additional  cost  estimating  requirements  are  prescribed 
in  AFI  63-101 ,  Acquisition  and  Sustainment  Life  Cycle  Management (15). 

False  positives  and  SDS  sensor  failures  will  drive  additional  maintenance  actions 
including  gaining  access  to  the  target  structure,  performing  conventional  NDI,  repairing 
the  SDS  sensor  system,  and  restoring  the  aircraft  to  flying  status.  The  potential 
economic  risk  presented  by  the  projected  false  positive  rates  must  be  included  within 
the  formal  Cost  Benefit  Analysis  conducted  in  this  task. 

Task  4  -  Airworthiness  Certification 

The  responsibility  falls  on  the  Program  Manager  (PM)  to  establish  a  certification  basis 
for  all  aircraft  modifications.  This  requires  that  the  PM  obtain  and  maintain 
airworthiness  certification  for  aircraft  under  their  direct  management.  Airworthiness 
certification  for  a  given  USAF  aircraft  system  is  managed  by  AFPD  62-6  which 
“establishes  the  policies  for  formal  airworthiness  assessments  to  ensure  that  AF 
organizationally  operated  aircraft  are  airworthy  over  the  entire  life  cycle  and  maintain 
high  levels  of  safety”.  This  policy  establishes  a  Technical  Airworthiness  Authority 
(TAA),  designated  by  the  Commander  of  the  Air  Force  Material  Command,  to  provide 
independent  airworthiness  determinations.  The  TAA  is  supported  by  an  Air  Worthiness 
Board  comprised  of  Major  Command  (MAJCOM)  representatives  and  subject  matter 
experts. 

The  criteria  for  establishing  an  airworthiness  decision  is  contained  in  MIL-HDBK-516B. 
Airworthiness  determination  is  based  on  an  assessment  of  system  level  effects  and 
risks  posed  by  potential  modifications.  It  is  critical  that  specific  attention  be  given  to 
system  level  installation  effects  and  potential  interactions  particularly  if  Active  (Type  II) 
SDS  systems  are  proposed.  Justification  for  airworthiness  approval  requires  verification 
that  all  applicable  airworthiness  criteria  are  addressed;  therefore,  the  veracity  and 
documentation  of  activities  conducted  in  Tasks  1  and  2  are  vital.  Failure  to  establish 
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technical  justification  (Task  1 )  or  demonstrate  an  acceptable  risk  (Task  2)  for  a 
proposed  SDS  solution  will  ultimately  fail  to  meet  with  TAA  approval. 

Task  5  -  Implementation 

As  previously  stated,  USAF  aircraft  modifications  are  managed  by  the  Modification 
Management  process  defined  by  Air  Force  Instruction  (AFI)  63-131  and  the  associated 
MAJCOM  supplements.  Although  a  detailed  analysis  of  the  required  processes  defined 
by  this  AFI  is  beyond  the  scope  of  this  paper,  the  tasks  described  above  directly  support 
the  implementation  strategy. 

The  PM  will  be  required  to  develop  an  implementation  strategy  that  clearly  defines  how 
the  modification  program  will  be  funded,  developed,  tested,  produced,  fielded  and 
supported.  Careful  consideration  must  be  given  to  system  spares,  maintenance 
technical  data  and  training  of  field  maintainers.  A  schedule  for  implementation  must  be 
established.  All  permanent  (P)  modifications  will  require  approval  through  a  Change 
Review  Board  (CRB).  Justification  for  approval  will  be  based  on  the  documented 
results  of  Tasks  2  through  4.  The  approval  process  will  vary  depending  on  the  using 
command;  therefore,  definition  of  a  comprehensive  implementation  plan  very  early  in 
the  program  as  well  as  close  communication  with  the  PM  is  key  to  successful  execution. 
Ultimately,  CRB  approval  for  permanent  SDS  installation  will  require  careful  execution 
of  all  Tasks.  SDS  modification  programs  should  only  be  presented  to  a  CRB  when  a 
sound  technical  justification  is  established,  there  is  a  clear  and  compelling  cost  benefit, 
the  solution  provides  acceptable  risk,  and  airworthiness  certification  is  obtained  from  the 
TAA. 
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5.0  CLOSING  REMARKS 


SHM  and  more  specifically  SDS  systems  for  monitoring  the  structural  integrity  of  USAF 
aircraft  may  provide  viable  alternatives  to  invasive  inspections.  The  success  of  and  the 
decision  to  implement  SDS  solutions  ultimately  hinges  on  the  capability  of  the  system  to 
reduce  the  risk  of  structural  failure  while  providing  economic  benefit  in  terms  of 
maintenance  cost  savings  and  aircraft  availability. 

The  proposed  task-based  approach  described  above  defines  a  logical  path  for  SDS 
system  qualification.  It  comprehensive  in  nature  and  is  guided  by  existing  USAF  policy, 
namely  Air  Force  Instruction  (AFI)  63-131  and  MIL-HDBK-516B,  and  is  tailorable  to  the 
application. 

It  is  anticipated  that  formal  guidance  in  the  form  of  Air  Force  Instructions  or  standards 
will  be  required  to  direct  the  transition  and  qualification  of  activities  in  the  future.  As 
SHM  and  SDS  systems  develop  and  mature  so,  too,  will  the  methodologies  for 
characterizing  their  performance.  Therefore,  any  future  guidance  should  reflect  the 
knowledge  base  as  it  evolves. 
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